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ABSTRACT 

The University of Wi scons in~Madi son is using a novel 
assessment process to find whether rising juniors have the 
quantitative skills needed for success in their chosen upper-division 
courses. Sampling from departments across the campus, information is 
gathered about quantitative skills used in specific courses and the 
extent to which students can show these important skills at the start 
of the semester. Instructors play a key role in helping to design 
f ree-response tests reflecting capabilities expected of students from 
the first week and essential for success in the course. Two important 
characteristics of this form of assessment are direct faculty 
involvement and close ties to student goals and backgrounds. It was 
found that the reflection, contacts, and dialogues promoted by this 
form of 3 sessment are at least as important as the test results. 

This paper briefly outlines the assessment procedure, highlights some 
findings about instructor expectations and student capabilities, and 
describes a variety of ways that the program has had a local impact. 
(Author /MKR) 
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Linking Faculty Expectations and Student Goals to the 
Assessment of Quantitative Capabilities‘ 



William O. Martin 



Abstract 



The I'nivei'Nitv of Wisconsin-Mailison (l’W-M;ulison; is usiny a nos cl assessment process to lind ss heiher einei>!ini: juniors have ilk, 
tjuanlicuise skills needed for success in their chosen iij»pei-di\ision courses. Sampling: from depaiimcnts across the campus (courses have 
included Fntiaplvs of Atiwrnsmg, liKijiliysmil ( hcntisiry. aiul Cin tai Analwsis) information is gaihered about (a) quantitatise skills used in 
spccitic courses and (b) the extent to \sliich students can shoss these imponaiit skills at the stall ol the semester. Instructors play a key role in 
helping to design free response tests retlecimg capabilities expected of students from the first week and essential for success in the course 
Tsso important characteristics of tliis t'orin of assessment are (a) direct faculty involvement aiul (b) close lies to stuileni goals and 
backgrounds. We have b>und iliai the rclleciion. contacts aiul dialogs proiiHUed by this tomi of assessment are at least as imponant as the test 
results This paper I>ricl1> outlines ilie assessment procedure, highlights svmie liiulings about insime!t>r expectations and student capabilities, 
and ilescribes a \ariet\ of \\a\s that the pii)gram has had a local impact. 

Assessment Perspectives 

Assessment in higher ecluration. Assessment in mathetimties used to mean tests aiul eraues. In recent years, 
the kind of assessnicMit that occurs in classrooms has receised increasing attention. Tlie National Council of 
'leachers of Mathematics has led the push for a broader conception of mathematics assessment with the 
development of Assessment Snmdanls ftn- St hool Mathematies ( 1993). Indicative of assessment's current high 
protile, I:\ving (1994) expressed uneasiness in the Amerieon Mathemaiieal Monthly about the lack of precision in 
the diseussii>n of “something so obviously sensible" as assessment, I expect that many others in higher 
eilucation, including mathematicians, share his uncertainty about the growing preoccupation with assessment. 

Assessment at the institutir)nal and state levels became an itnportant issue during the 19S0s (l:well and 
Lisensky, 1988); assessment programs have nmv been mandated fm- colleges and unixersities in many states 
(I'weli, 1994). Accrediting agencies — for example, the North Central Association of Schools and Colleges that 
accredits the UW-Madison — also have introduced assessment requiretnents; so, assessment beyond that carried 
oiit at the course le\el will increasingly require faculty attention. Mathematics departments, because of their 
important service function for undergraduate degree programs, will have an extra role to play in campus 
assessment programs — this besides their need, like all departments, to assess in the major. 

This paper is about the evolution of a (.luantitati ve assessment project, run by faculty tnembers in mathematics 
and statistics, designed to meet external assessment rcijuirements in a way that is usetiil locally tor participants. 
An important charaeleristie of the project is its focus on spccitic needs and expectations of students and faculty in 
particular courses. The assessment project operates at an intermctliate lexel between the individual classroom and 
tile institution, 

iivaluatitm and assessment. ’Fhe terms iiwes^tfienl ami evaluation are sometimes used mtcrehangcably . 
though there is some distinetion between them. Angelo (1994) tlctincs assessment as “a means tor focusing our 
collective attention, examining assuiuj'ttions. ami eieatmg a shared eultuie dedic;iied to mulerstatuling and 
continuously improving the quality ot higher learning" (p. 1 ). ’I'lie iiuantitative asst ssment project at the 
I'nixersity of W'iseonsniMarltson ( rW'-Madison ). treats a\u‘\\nu'nt as the gathering and reporting ot inlortnation 



' This jiaper xxas prepared tor llie MAA session on \cw Dim tions m .Stiulent Assessment at ttie Joint 
Matliematies Meetings ot the MAA and AMS in San I'nmciseo. January 3-7. I99.S. ('opies of (he paper and 
additional inlormatimi are axailahle from the author at North Dakota State University. Department (»t 
Matliematies. 9)0 Mmard. P() Hox 507'^. l argo. ND .^SIO.^ .^075 (email; \v imartin(^' plains.nodak.edu). 
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about stiklents* ijiiantitalivc capabilities without the Judgement or valuing of those iindings that would be 
connoted by vvalnation. To illustrate the distinction, much classroom assessment in mathematics contributes to 
evaluation when instructors use inlbrmation collected on tests and homework to assign student grades. 

Quantitative assessment at Madison began for the most familiar reason: it, along with verbal assessment, was 
externally matidated by the Governor of Wisconsin and the Board of Regents. Amid increasing pressure for 
accountability in higher education (Hwell, IWl), all system institutions were directed to have programs to assess 
the i|uantitative and verbal capabilities of emerging Juniors opeiating by 1991. Although the impetus, and some 
sui')port, for assessment was external, the form of implementation was left up to institutions. 

The concern for faculty in\'ol\’cinent in and control of assessment tnight have the appearance o\' self-interest. 
Nevertheless, the importance of faculty lnu)l\enient is not Just a local, faculty concern; Ferren's (1993) call for 
institutional assessment to be faculty -dri\en is a common theme in the assessment literature. Scholars working 
on assessment have noted the lack of educational impact of the widespread use of standardized tests for 
accountability purposes. For example, the e\tensi\e use of \arious standardized tests tor many years at the 
Ihiiversity of Kentucky-Knowllle appears to have had little impact on instruction or learning (Banta, 1993). 

Lack of real faculty oi student engagement in assessment may explain the missing impact, Such results pros ide 
useful lessons h.)r future assessment efforts. Our own tindings at Madison agree with Banta's obser\ ation: 

I:\periencc o\er the jiast decaile with assessnicnt at the postsccondary le\el has indicateil that tlie tiiuling 
or results obtained from assessment are less important in stimulating impro\ements in practice than is the 
process of bringing faculty together to discuss purposes, student outcomes, and methods of instiuction as 
they prepare tor outetunes assessment (ILmta and Fisher. 19Sh). (p. 51) 

Altliough c|uantitiiti\ e assessment at Madison grew out td' institutional assessment and is important at that 
level, the focus of this report is its utility for faculty, students, and departments, a perspective that has 
recei\ed much less attention than lias classroom and institutional assessment. The assessment procedure we use 
helps to identity tlu‘ c|uantitati\e needs of stialents, expectations of faculty, and goals ol ;.lepartments. The 
inlorination that is generated can help faculty members and departments to improve the instruction and lear»’'* ig 
of undergraduates (not only mathematics anil science majors) at the institution. 

Assessment Ohjectives: Identifyinf; and Meetinf; i\eeds and Expectations 

Do students enter college (or graduate school, calculus, upper-di\ ision courses) with skills lequired tor success? 
Do grades in prior courses accurately rellect student mathematical capabilities? Fiow go<xl are graduating 
majors,^ Do technologies (or writing, group wtirk. hard problems) really make a difference in mathematical 
learning.^ As tCiichers and scholars, wc have beliefs about these and similar questions; assessment is a tool to 
ensure that such beliefs are supported l\v more than intuition. 

Assessment has historically been an important part of higher education. Sew (Iireeti(>n\ in (/.s.ve.s,s7;u7// reters 
both to a chance of methodoKycies used in the classrocnn and an awareness that assessment is important at other 
levels beyond indi\i;.lLKd courses. With se\eral notable exceptions (such as college entrance examinations; 
statewiile examinations such as the New ^'ork Rciients' examinations; and graduate qualifying examinations in 
doctoral institutions), much less attention ha^ i'leen gi\en to assessment of the impact of programs and sequences 
of courses, Americ,m educational trailitioiis most notably, the tradition of local control td education -may 
account te.r the lack of .isscssment at this lc\el. In contrast, assessment at a program or institutional level is an 
important feature m niiiny other nations (tor example, in I'uropc.in. Asian, aiul l\icilic countries), 'riiere. 
compreheiisi\ e exieina! examin,itions lu\e significant eonsei|uenees tor students, faculty, and institutions 

In the iihsence of extern, il iissessinents. the se.irch for ways to monitor the impaet of progiiims and iiistitutn>ns 
h,is turned both outward to coimnercial testing seixices ilianta. F)^)3. p lists four mam insnumeiits ) and 
mw, ud to tfie mstiiiition itself Within <\ ^.I'llegc oi um\eisity. three constituencies lu\e ,i diiect inteicst in 
(|iiantita‘i\e assessment: (a) students, ih) mathcmaties faculty, and (c) faculty in other "client” depaitments 
l;ach is likely to hii\e unique leasons for and to use a ses>ment diffetently To he loc,ilh woithwhile. 

,issessnient ^hould address the needs of e,ich gioup 
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Student needs expectations. Students pkiy a central role in assessment. Often, though, their role is simply 
as a source of data: students take the tests, complete assignments, or are interviewed. Traditionally, they do not 
use the data they generate. Assessment can serve a broader purpose for students than just a tool tor evaluation 
and certitication. They liave expectations and goals that they bring to college; these may change as they progress 
through their degree program. Mathematics curricula retlect what faculty believe students need to know. To 
what extent do students provide information about their own perceptions of needs or the extent to which these 
needs were met b> mathematics courses or programs!^ Mostly, not at all. Are the \ iews of students important? 

Or should they accept what is offered by educational experts’!^ I believe that student \ iews are \ ery important, 
especially as the\ re\eal differences between the intcfulcd and the pcri'ci\ i‘d or received curricula. 

Mathematics faculty expectations, it is interesting that faculty members in a subject founded on deducti\e 
rationality can show a w illingness to base curricula and teaching on personal experiences and intuition instead of 
findings from educational research. A useful exercise is to think about the range of iiuli /idual and departmental 
innovations or changes in mathematics o\er the past decade. How many were based on careful analysis of 
institutional or departmetUal goals and data about the extent to which these goals were being met.^ How manv 
reforms were formally e\aluated? What is knowi (as opposed to hclicvcd) about the impact of changes on 
students ? I am fascituitcd by some of the thoughtful electronic discussions that take place on the American 
Mathematical Society e-math ct/A discussion group; I have also been struck by the lack of eviiience cita 
111 these discussions to support c\cmi [dausible claims about the benefits of instructional approaches. As 
mathematicians, we should discuss and specify our own goals for (undergraduate) courses and assess the extent to 
which thc> are being achiewed. 

Client department faculty expectations, l‘acult\ intmibers from across a campus expect students to come into 
tlicir courses and programs with certain iiuantitati vc capabilities. We have found that in non technical Ci)urses 
faculty nia\ initiidK sa\ the\ do not use an\ quantitative material: during further probing, wc often discover that 
they have a varietv of basic quantitative expectations that they hatl not consciously lieen aware of (e.g., using 
percentages, reading tables, and interpreting graphical representations). Most mathematics faculty members have 
iicard comments or even complaints about the mathematical capabilities of students in other departments; perhaps 
even they have had similar feelings about the mathematical preparation their own students bring from high 
schotd. W'hat capabilitic’. do faculty in otiicr departments need from their students ? Do mathematics 
[ircivquisiies match these expectations I)i^ students have the necessary capabilities when they get to tlic 
courses ' It seems tluii much of the information about these questions is anecdotal. 

Externa! constituencies. In return for their financial support, parents, taxpayers, legislators and others have 
shown a desire for evidence that supports claims made for higher edueatimi. A problem w ith external calls for 
accountability is that the impact of higher education is not easily measured; measures that are easilv 
cvmiprehemietl and compared, particulatly from standardi/cd tests, may bear little relation to the goals of the 
participants in a particular institution. 

How can information from such diverse perspectives be collected? Madison's assessment phiii i designed to 
meet external assessment requirements meaningfully, sinuiltaneously providing useful information for student and 
faulty participants. The focus is on emerging juniors. Included in this group are students who last studicLl 
mathematics in high schot)l along with those who have completed two years of extensive \tudy in [ihvsical 
sciences and mathematics, (‘leatiy no staiulardi/ed instrument could match the ijuantitativ e backgrouiuls of ail 
juniors on th s campus, In our proiect. tlie link bet-vecn faculty expectations and stiulent backgrouiHls is maile bv 
tying assess! lent t<» suulent course choices at the junior level 1'his is when stiulmt^ move from geiieial 
education to uj^per-div ision specialization m the maitu field Patterns of results .icross a range of courses {'»u)vide 
evidence about wh.ethei stiulenis on thi'' campus geiicrally liave develojH'd ;ideqi.iate quantitative land vcrinil) 
capabilities to deal with iIk“ specialized work ni tlu' iiia|oi Hecause students choose ma|oi piograms and couises. 
our assessment lellects their goals and hackgtoiinds 

Quantitative Assessment: Gathering, Or^anizin^, and Disseminatinf^ Information 

\ssessinent begins with questions Once the purpose of assessment that is, the audience and icasons foi 
assessment lias bt'en identified, one must Llecide what mfoiniation should be collected. When we talk about 
how conises or pioeiains coiitnbutL- to leatmng. we a‘-siinie imphcitly at least. s(une goah. o\ expectations against 
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which U) measure achievemeni or progress. Course descripiions include inl’ormaiion about goals, as do 
departmental statements (such as might be found in college bulletins). Commonly, such documents focus on 
what will be covered in a course rather than the capabilities that students will develop. As guides for assessment 
of the impact programs on student learning and knowledge, these statements of goals have little value because 
they either are closely tied to individual courses or are too broad and content focused. Students study 
mathematics for myriad reasons; other departments require mathematics for varied purposes; all faculty members 
do not even share unique goals for undergraduate programs (Boyer. 1990) or, in particular, mathematics. All of 
this further complicates the articulation of ohjectises for assessment, 

Our assessment process begins by choos g junior-le\el cum es from departnteiits across the caitipus. 
Instructors are asked to identify the quiuitita \e capabilities stud mts will need to succeeil iti their course. With 
their help, we design a test of those skills. Studetits take the II -.: carls iti the setnester; correcteii papers and 
infortiiation about class perfortnatice are returned within a week. 

Test construction identifies goals. Identification of goals is built implicitly into the Madison assesstnent 
process; faculty articulate qiianiilative expectations that are closely related to specific courses as they help us 
construct the quantitatise readiness test for their course. As they do this, instructors idetitify the quantitatise 
skills students need to be successful during the setnester. Bs' design, our tests reflect tiiaterial that students will 
use during the semester, content that the instructor dties not plan to teach and that students should already know. 
This is not generally an eass task but the attempt to identify specific, necessary capabilities, as opposed to a 
more general **w ish list.” is one of the most valuable parts of the assessment exercise. 

Motivation is a key issue. A significant problem w ith assessment outside the context of a specific course is 
getting students (aiul faculty !) to participate seriously. Our assessment methtid is designed to be useful to facults 
and students; getting iiidis iduals to focus on specific need; and expectations is what gises meaning to the test 
outcomes. The altitude of instructors is crucial li^ the mccess of the exercise. W'e empliasi/e to participating 
faculty members ihiit the via\ tlic\ portray tlw tc\t to \ttulcnt\ is the most important factor in whether the 
students' efforts on the test [Tn)\ide useful information for atnone (including the institution), bew' stuilents will 
buy the "Do this fur tlie good of futuie sUideiils and the iiisiitutioM” line. The best approach, we ha\e found, is 
for the instnu tor to tell suiiients 

► The test tloes not count toward their grade, hut 

► Test results will help these students know their quantitiUise readiness for the course early in the term 

► I'lie instructor is \er> interested in how the> do. so it is crucial that students try their best 

► Kesulls of the test may lead to coirrse modifications to match content to student Ciipabilities better 

Ob'.iousls. the instructor needs to believe this there is little to be gained b\ Using to assess in a course where 
the instructor does not, so s\e dti not (naturalls, sse do irs to consince skeptics since it is not uncommon for 
facults to doubt the salue of assessment imlialls ) Although the process generates information that should help 
improve courses and programs across the campus, the focus in each course is on the immediate benefits for 
PiUticipaling students and inslruclcMs. 

\N’e base deselopi.'d a reliable coding schctiic that idlosss niatltematics graduate students to record information 
about student perfornKince on scantron sheets for latci anaissis so that ciurected lest palmers can be returned to 
students within a week, (riiidets rate tiie degti'e of success ft>r each problem using a tise-point scale, lhe\ also 
code infonnatit>n about the steps students take to\^ard a smIuIioh (e g., differcntraied coriectl> or desised an 
appropriate represent<ition ) W'c retuni the corrected test papers along with test solutions and references to 
textbooks that could be iiscil loi tcsiew 

Although we d»> compute (but di» not icpiut to students) ,i te^t scoie lor e.ich papei (the numivr t>f piaddems 
dies had completels or basicalK collect), the mam focus is on tlic pmpoition of the chiss that could do Ciich 
pM»blcm rills mloimation is cleatls useful lor iiisiructois. across a series (d courses there arc patterns in the 
tesirhs that pm\ iile useful inlormatron for departments, colleges and dis tsions. and the institution Our 
methodologs adtiresses the problem of how to match assessment to student backgrounds bs linking assessment to 
courses selected In students as thes begin work in their niaioi .Still, esen within a course there can be 
corisideiable saiivibrlils in student quantilathe backgrounds We obt.un infonnation from uniseisits records .iboiit 
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the maihomalics and statistics that students have lakeri. Without identifying individuals, we report this 
information, along with the assessment test score, to course instructors. 

Discussions with faculty. Faculty contacts are central to this form of assessment. The validity of our 
findings is dependent on instructors ensuring that the test we design accurately reflects the quantitative 
prerequisites for their course. For the assessment to have local value it is necessary that findings are circulated 
among and discussed by individuals and groups with an interest in students' quantitative capabilities who can 
respond to the results. This is at the crux of the Madison assessment strategy, and is an important contrast with 
standardized testing. It is worth emphasi/iiig that the main advantage of' this approach is in the ongoing ilialog 
about student knowledge and learning that is promoted, indeed required, to conduct the assessment: 

► Individual facultv members must focus on specific expectations for a course to prepa*e an appropriate test. 

► Student needs and backgrouiuls are refleclCLl in the assessment process because the test is lied to a course the 
student has chosen, usually at llie start oi' their siuilies in the major. 

► laieully from mathematics. Niatislics, and client departments talk about faculty expectations, student necLls. and 
student performance in relation to specific courses aiul programs. 

► The conversations are tightly focused on the realil) of existing course content and written ev idence from 
sUufenls ahiHil their Lji.anliiativ e capabilities. 

► I'vervmie involvetl. sljdcnts and fac ultv. gains iisctul information that has immcLliale significance apart froni 
its broader, long-term institutional meaning. 

rindinffs 

Our assessrner.l work has pn)duced some smpiiMiig lesiilis. thought it lias ru)i generallv revealei.1 large 
discrepancies between mstructor expectations aiul student capabilities. Instructors often want students to be able 
to reason itulependen.’dv. to make interjirelalions aiul to Llraw on basic L|uaniilalive concepts in their courses; they 
seem less concerned about stialent recall of specific techniques. SliiLlents. on the other hand, are more successful 
vv ith routine, statuiard comjnitational taskN and often shovs less ability to use conceptual knowledge or insight to 
stdve less stamlard quantitative [Problems (Haunian aiu! Martin, to appear). Several [irtiblems will illustrate these 
biulings and the nature tif our tests. 

Instruclois of courses that have a CliIcuIlis j^reiequisiie often want students to understand what a Llerivaiive 
represents; diey are usuailv not interested in student recall of differentiation lechniL)ues or formal limit ilefinitions. 
Two problems designed to probe student undei standing of derivates have been chosen by instructors for use in 
in. my cinirses 




Problem I ! ivjUi I cu//>/o o/ (/ f (Oiil i!s fust iind ncco//(/ (/ cMi cv. / iinJ 

f I tihi'l i (it h i !u \ r iht lant I'l: >n ot stt ttnd dcnvonvi Ididoin \<'m 
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Problem 2 h'i^iiiv 2 I'/i c.v ihc ^niph at d tunclion y ~ J{x) . I '.se the yrupli to answ er these 
(jiu’sttnns: 

id) i'.stiniatc f'{A) conrct) 

{/j; Estinmtc f {2). i44^'( 

ii ) On wliu li inicruili M it anw docs it appear tiuit /^(x) < 0 / (65^'f correct) 

(The percentages are the proportion of students in a recentl> assessed eneineeriiiL’ course sstio aiiswereil 

the question correctly — the course prerequisite ^^as three semesters of calculus) 

hew students with onl\ one semester ot calculus have answered either problem correctls on our tests, 
although the material is drawn from the eaii> part ot liist semester calculus, l;ven in classes where students have 
completed the regular three-semester calculus se(|U>:nce their success rates are surprisingly low for these 
introductory problems. h‘or example, about three-i|uarters of the students in on class correctly labeled the three 
curves in prohlent 1; under one-third of tho^e students could adequately justify theii labeling. The students had 
reasonable matliematics backgrounds; More tlian half had a M or I'letter in their previous mc.thematics course, 
which was either third semester calculus or linear algebra, .lust live of the S7 students had a i) or \' for their 
jnev ious niatliemalics course, Success rales are higher it w e just ask them to diflerentiate or integrate a function, 
l or example, in the same class ovci lee-quarter'" of the students correctls evaluated j te ' dt. 

These results jMovide useful infoiiiiation about both (ai lacultv expectations in other departments (and in 
upper level mathematics courses assessed as part of the j'n’ogram) and (b) student capabilities in relation to those 
expectations and various mathematical biickgrounds. We have found that most junior level courses have one of 
three kinds of expected backgixuinds; (a) no college nuilhematics, (b) one semester of calculus and perlups a first 
statistics course, and (c) the full three-semester calculus sequence with some work in dilferential equations. 
.Although the iiist level ot expectations does not require collegiate mathematics, man\ instructors iio expect a 
cert.un level of quantitative lileracv from school mathematics and statistics. 

The purpose of this pa|'>er is not to re|M>rt our lindittgs, some of which have been leported elsewhere ilTiuman 
and Martin, to appear). Individual tiiulings, such as those given above, have local signilicance, but we have no 
icason to believe ihev ,uc generali/able bevond specific ccuirsc^ or perhaps our owm institution. ’The signilicance 
of this vvotk hu' others lies in the melhodologv tor investigating the match between facullv expectations aiul 
stiideiit capabilities Hccaiisc each assessment is v loselv tied to a spci.itu' cmir^e, the assessment's impact is 
olteii iiaiiovvl_\ lociised Surpt isiiml\ . though, some tindings have even had a campii- wide effect on the 
urulergiadmite currK iiliim 



Impact (pf the Assessment Project 

A question .ihout this, m laci anv. assessment piogiam is W hat is its imp.ict ’ How can this make a ddleiencc 
and lead to improvetuent * .Alter all, it is well known that students have difluulties with qiianttlative skills and 
leasoning How does assessment guide iis in the luluie ’ Such questions come from a view of assessment as 
siiinmative. evaluative, .uul ludgmental something external that occuis ,it the end. that points to success or 
lailuie. It Is a h'pdovvn view, with assessnts making’ the iudgements and being responsible lor lecoinmending 



W'll.I.IAM (). MAkl lN 



A^SIiSs^MliNl ; l-ACn/lT l:\lMiCTAl IONS AM) S lVDliNT Gi h\lS 1 



dian^ies. Our view of asscssmenl is Lliffcrcni. Our work has liad an inipacl, bul In a broader sense than 
suggested by the forgoing questions. 

The model for this project comes from a conception of assessment as an integral, ongoing part of education 
that hi\s formative and summative characteristics. It is based on the idea that faculty members are best able to 
respond to information about student capabilities: what they might need is assistance identifying their own 
expectations and the relevant knowledge aiul skills of their students. This form of assessment is designed to have 
an impact by promoting reflection and by encouraging curricular aitd pedagogical ilecision-making based on 
knowledge rather than on assumptions and intuition. 

It is worth noting that recent mathematics education research in elementary grades has shown that teacher 
knowledge is an important distinguishing characteristic of the type of learning that occurs in classrooms. Content 
knowledge and pedagogical cimtent knowleilge are important, but pe !.ui)s iiK)st significant for college faculty is 
the idea that effecti\e teachers have a good sense of svhat their ' .^dents know and use this knowledge to guide 
their instruction (see l-ennema and laanke, 1992 for a discuss^ . m of the impact of teachers' knowledge), 
Assessment should help faculty lo be attuned it) the know!e<;ge and capabilities of their students in relation to the 
demands of their course, thereby fostering instructional imr osemenl. 

Impact at a variety of levels. Instructors have mostly reacted very favorably to the assessment process 
T.iose who do not report making any changes either found from the tests that students had the prerequisite skills 
or said that they were alreadx aware of the dilliculties ami had modified their approach to deal w ith them — the 
project simpls confirmed what the\ had suspected. In cases where instructor expectations Lliffered from the 
results the\ often reported making changes, either by omitting reviews that no longer appeareil necessary or by 
including additional wDik to de\elop imporlani, missing capabilities. 

Students report less influence of assessment. parll\ because mans mistakenly see it as a pretest of material 
that will he stiulied in the course. Others tail to see the connection between a mathematical problem on the test 
and the way the concept is used in the course. I‘or example, the lest may inclmle an item in\’ol\ ing the concept 
of definite inicgra! as area under a curve— in the course, students may use the concept in their work with 
frequency distributions without recognizing the connection to the test problem, in technical courses, manv 
students (perhaps around half the class) reporleil suid>ing both before and after the assessment lest and said that 
the ie\iew- was useful. Sometimes, unfortunately, students made comments tin our folloW'Up questionnaires such 
as "Waste of m> lime" or "If I wanted to he a math major I would have taken a math course." Most students, 
when questioned at the end of the semester, recogni/.ed that the skills were imporlani in their course but had not 
chosen to use assessment intormalion to help prepare for those reL|uiremenls. Students in more technical courses 
are more likel\ to make comments such as "Helped to shock me into relearning some calculus" and "The written 
ctuTeclions — including specific reasons why I did not reach the correct answer — were greaily appreciated!" Such 
reactions shm\ that some students do, in fact, find the diagnostic exercise useful. 

Departmental impact. The value to the Department of Maihema'ics of the data generated by assessment is 
quite clear We report annually to the entire faculty, hut we have probably had greater curricular influence by 
targeting our findings at individuals and comniillees responsible for specific levels or groupis of courses, 
particularly jirecalculus and calculus, Tindings from many assessed courses have shown, for instance, that faculty 
want snulenls to interpret grapliical representalions. I'his had not alwa\s been einphasi/eil in mathematics 
courses. It was somewhat ironic, but instructi\e. that in an early meeting to discuss our findings with a 
curriculum gmup in malhemaiics iuie facull\ member remaiked about a pioblem 2, "rm not surpiised students 
i.auihln'l ilo that I nc\ci ask such questions in m> clas> " A eiilleague immediatel> resp<mded that he thought 
such tasks were \vv\ important and alwa._\s emphasi/eil such ideas when he taught calculus. OlmoiisK (‘Ui 
assessment work Ciin siiimiliite \aluable ilisLii>sions about wlial is aiul slumid be co\ered in miilergiiUluate 
iiKiihemalics <.oui scs 

Our findings about giaphical lepiescntations ha\e led couise cot>rdimitors to encouiiige instructors to give 
iiKieased aticntiou to giaphical icprcsentations of funciions IVihaps more imporlani, though, is what our work 
shows about the kind of mathematical skills needed in other departments. Fhose instructors seem less coneerneil 
iibout coniputalioiKil. algorithmu knovvietige than moie lOiK eptiial. problem sols ing capabilities 'I'liis has 
implications not |Usi for the lon < >’/ of m ahemaltcs couises, but aUo foi the \\n\ that mathematics is taught 
including expectations for what si idenis uml ' 
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Assessment has intUieneeJ participating tlepartments. To help ensure that assessment results are seen and 
discussed beyond the individual course, we produce a summary report tor faculty members in participating 
departments. After distributing this written leport, members of the assessment committee attend a regular faculty 
meeting to ans er questions and discuss the issues raised by assessment. The information we provide could lead 
to a variety of departmental changes (so, not all quantitative problems arc the responsibility of the mathematics or 
statistics departments), Se\eral examples will highlight the impact of our work has had on t)ther ilepartments, 

► After (inding that mans students in an introductory course were unable to handle material from calculus, one 
department increased the prerequisite from tirsl semester business calculus to two semesters of the regular 
calculus sequence. They did this not because the students needed the additional content, but to ensure that 
their students had further deseloped the necessary fundamental ideas by using and resiewing them m later 
mathematical work. 

► In another department, many students had jioor records for their college mathematics courses. During the 
facultx meeting at which assessment outcomes were discussed, a faculty member remarked that students 
claimed they did not reali/.e the> would he expected to km>w material from a prerequisite calculus course in 
their later course work One response to this problem in\ol\es student ad\ ising. csjieciall\ for first- and 
second-sear students as thc\ meet general education requirements, b'acults' adsisors should emphasize that 
prerec|uisite courses coser inij'>oitant knoss ledge that will he essential later; that prerequisites arc not |ust 
additional credits that st^rse as a hurdle on their path to the degree. 

► baculty members in other departments tspically welcome the interest of our committee, with its mathematics 
and statistics facults members, in their quantitatisc expectations of students. Often. the> sax that this is an 
imp(Mtant area that the\ ha\e neglected in the past. Recentlx. one (non lechnical) dejxartmcnt included 
discussions of quaniitatisc difficulties i)\ their stu'dents as the\ restructured theii' undergraduate program, 
deciding to incoiporate more qt.iantltati\e reasoning work in their ow n lower lexel cauirses. 

► In another department, following a planning session \xith our grouj') for an upcoming assessment, the 
coordinator for a large introductorx science course remarked that he "couldn’t remember haxing spent even 
live minutes discussing the sjx’citic quantitatise needs of students with colleagues" during his years (decades) 
at the universitx Mhe couise had a calculus p’erequisite ). We were gratified that the facultx members we 
were working with lecogm/ed tuie ol our program’s nuwt important giials and the xaluc of this form of 
aNssessiiient. 

Campus-wide impact itf assessment. ,\n ear!\. striking finding from the assessment jxroject xxas that some 
students were actixely axoiding aiix courses with quantitatixe exjiectations. These students xxerc unable to 
complete basic (|uantitatixe literacx tasks sucb as using percentages and extracting information from tables and 
bar graphs. A university curriculum committee saxx these results and later recommended that all baccalameate 
degree programs include a six-credit quantitatixe requirement (before this, it xxas possible to get a R. A xxithout 
anx collegiate mathematics tir statistics). 'I'lic rcconimeiulation xxas adopted bx' the I'aeullx .Senate, a clear 
indication that our focus on iiulixidual couiscs can produce information useful at the broadest institutional levels, 
I'auilty responses to assessment, Ihux dti facultx resptmd when main students do not luxe necessarx skills, 
tpiantitatix e or otherwise ’ Soinetinies. we luxe found, xxith levignaiion: "It would be loxelx if we required three 
or even two semesters of calculus. Rut tme will luxe to do." was the res[Hmse from one instructor, articulating 
the coiistiiiintN that applx to prerequisites in degiee pingrams ,\inwher facultx member said he had chosen to 
ieaxe quantitatixe imilerial eompleieix tut o\ hiv ikmi technical course because (a) students lacked the necessarx 
skills ,iiul (b) he had plentx <d other mateiial to coxei 'I'liis was uist the \wos\ extreme retii.tum that xxe luxe 
eneoiintered to the problem of students lacking quantitatixe Ciipalulities, Other faeultx membeis luxe iiKo 
leported "walering down" quantitatixe exju’ctatiiuis in ct'uises because td j'icrceixed student weaknesses Ibis is 
a disturbing finding, and one that caniud be easilx aiUliessetl bx mdixiduals since students can "opt out" ol 
ciuirses xxith high expectations. Our assessment can help to stem this trend by exposing the institutional impact 
of such indixiilu.d decisions U' facultx members and dc|>ai tmenis Some facultx cairx on ix'gaidless ( )ne 
piolessoi lenuiked tlKit "I lie students pick up |lhe necessarx skills) as xxe gi>. drop, oi peiish." .irticulating 
another lesponse to .ipparent weaknesse> 
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Shi>rtcominf^s of this assessment work. Angelo and Cross (1993), in their practical classroom assessment 
guide tor college faculty, suggest tliat assessment Is a cyclic process with three main siages: (a) planning, <b) 
implementing, and (c) responding (p, 34). Their view of assessment as on ongoing, integrated part of instruction 
is similar to that of our project. Although ! have cited several positive responses to our assessment work, there 
have also been instances wliere assessment revealed problems but no action was taken. This breaks the 
assessment cycle after the second stage. I expect this to be an enduring problem for several reasons. 

► Our appnnich is (.leliberately collaborative and non prescriptive. Because of this, faculty jxirticipate 
willingly — assessment is volunlaiv for indiv iduals because its utility depends on participants believing it is 
vvoithwhile. Our primary role is as information pi ov Iders; we help faculty members to identify e\pectati(’>ns 
and pn>\ide information iibout student capabilities anti backgrounds. Interpretation of and resptmse to these 
fndi-igs is left to those vvlu^ are affected (vve do. of course, responti to retiuests for suggestions or 
interpretations). 

► Not all problems have simple solutions. If stutlents have difficulty with the arithmetic of complex numbers or 
reading graphs, the problem can be atidressetl by inclutling material in mathematics courses. When 
assessment shi)ws that students have difficulty with concepts, the remedies are much less obvious. Including 
some new problems or even a unit will probably not be the answer. Still, one can hope that faculty 
awareness of student difficulties of deeper and more complex oiigins may increast.‘ attention given to tlie 
needs and capabilhies of students in individual classrooms. 

► Solutions to st)ine problcMtis do not rest with the institution, h'or example, a dej'iartment offered an 

introductory course tiiai 'eijuired a semester of calculus. We assessed in the course for several years, finding 
that most students wee unable to handle any tasks from calculus, liach year, as we rev ised the assessment 
tests the instructor reittoved more of the calculus material, reflecting his growing awareness of student 
capabilities. We knew that many students hail taken a business crdculus course instead of the more rigorous 
science and engineering version, so we suspected that business calculus was iku adequately covering the 
necessary mateiial. Our view changed dramatically during the tliiixl assessment, when we also gained 
information abtuit the students' backgrounds in mathematics. Many students had very poor records in 
mathematics; some had repeated mathematics courses two or three times ainl many harl low grades (C or D) 
in <aiculus. 'rhi>, information exjdained vvhv tliey had so much trouble with calculus a year or two later. ’I'he 

real s(>kilion was for students to work harrier to rlo better in prerequisite courses. 

W'e recogni.a’ that assessment is cyclic and that respoiuling to findings, especially of sliortcoinings. is 
important: we also know that not every j'lrohlem has a solution, laiculty members are able to deal with the 

educati()nal issues. They can do this effectively if they have information about the e\isti.‘g crlucational situalu)ii. 

Conclusion 

Assessment has always hatl a prtmiineni. it narrow, role m the study of niathcmatics in colleges and univc ities. 
I'AccjM for graduate qualifying examinations, most of this attention has been at the level of indiv Irlual courses, 
with assessment used to moniior student learning during and at the eiul of a jiarticular class. 'I’he natural focus o\ 
a mathematics taculty is (»n their majors and graduate students. .Still, their role in a college t>r univeisity is mueh 
larger because of tlie service they piovide !n tiiiining stiuients tor the quantitative demands of other client 
depa tents It is important that mathematicians monitor the iinpa.ct of this service lole along with their 
jtrogianis tor majors 

This pajter oi. lined a liKally developed [troL'eduie that addresses an iiiijtorlant but often neglected aiiiension 
ot assessment in maihematics. student retention ot mathematical knowledge over tlie longer term and the match 
between faculty expectations and student capabilities in subsequent courses. Recently. Selden and Selden (1‘I93) 
listed long teini letention ot mathematical knowledge as one ot seveiail important issues deserving study. 

Although this sort of assessment has reeeiveil little attention in the j^ast. it is especially worth pursuing in this era 
ot lefi'iin in school and collegiate itiatheinatics Naturally, quantitative assessntent also contrihutes to assessment 
at the broader institutional level, a need that has icccivcd increasing attention in recent ycais d'well. 19W|). 
Mathematics faculties Inoe an intciest m assessim' their woik lathei tlian leaving it to oulsideis with less stake or 
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inlerost in the subjecl, The approach reported here allows mathematics departments and faculty to yive more 
attention to this important aspect of their educational mission, 
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